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ABSTRACT 

The Minnesota Department of Education's Office of 
Community and Adult Basic Education commissioned an evaluation the 18 
adult refugee English-as-a-Second-Language (ESL) programs funded in 
1965-86 through the Minnesota Refugee Prograia Office of the 
Department of Human Services. The purpose of the evaluation was to 
identify effective practices and make recommendations for future 
decision making and planning. Two sources of information were used to 
explore this question: (1) the data system used by the Refugee 
Program Office; and (2) the information from the people delivering 
and receiving services. The report discusses the project design, 
methodology, and activities, and examines the clustering of students 
based on student performance levels (SPL) . The next section 
summarizes the context and method variables that are perceived to be 
most effective in teaching English to adult refugees. The final 
section of the report looks at the critical issues for public policy 
decisions. This section also includes recommendations in the areas of 
student performance measurements and the dat. system being developed. 
(GLR) 
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Dear Reader: 



In November 1985, the Minnesota Department of Education, as part of the interagency 
agreement with the Department of Human Services 1 Refugee Program Office, commiss- 
ioned an evaluation of Minnesota's adult refugee English as a Second Language (ESL) 
programs with funding from the federal Office of Refugee Resettlement. Outside 
consultants, Margaret Bringewatt and David Nelson completed the project report in 
May 1986. At the ne, there were eighteen programs operating in the state - 
twelve were ABE programs administered through local school districts; three were 
mutual assistance associations; one was a non-profit agency; one was a program 
for deaf refugees; and one was an organization responsible for the state-wide 
coordination and training of volunteer ESL tutors. 

The original purposes of the evaluation were: 

• To define and measure the effectiveness of the above mentioned adult 
refugee programs; 

• To identify successful classroom practices, and 

• To make recommendations that would guide state agency planning and 
decision making. 

As you consider the report, several points should be noted: 

1. During the design phase, the report's primary audience was to be staff of 
the Department of Education and the Refugee Program Office. However, as 
the evaluation process evolved, a second but equally important audience 
identified was Minnesota's ESL program providers themselves. 

2. Because the data system was in the process of being set up during the 
evaluation, it was impossible at that time tc analyze student progress 
and language level correlations. Consequently, the plan to measure 
effectiveness had to be postponed 

3. The issues section is not intended to be exhaustive, but rather it is 
offered as a summary of critical concerns to those who are familiar with 
the context. 
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4. The study was undertaken to explore the field's environment, opinions, 
and ideas. The state did not initiate the study to learn what planners 
and policy makers wanted to hear but to promote an honest discussion of 
providers' perceptions. Therefore, the views in the document do not 
necessarily reflect the views of the state departments involved., 

5. The process of conducting the evaluation was in many ways as important as 
the final report itself. By inviting full participation from people 
working and studying in programs, the evaluators were able to make 
adjustments in the design in keeping with the realities at hand. These 
adjustments were key to the success of the project. 

6. This report is not an end in itself. It is meant to serve as a point 
of departure for further study, discussion, and program development. 

Comments concerning the report should be addressed to: 
Diane Pecoraro 

Community arid Adult Education Section 

Minnesota Department of Education 

550 Cedar Street 

St. Paul, Minnesota 55101 

(612)296-7500 

We wish to thank all who participated in this project. Their efforts and insights 
are greatly appreciated. 



Jane Kretzmann 

State Refugee Coordinator 

Department of Human Services 



Diane Pecoraro 

Refugee/Adult Education Specialist 
Department of Education 
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OVERVIEW OF THE PROJECT 



A. INTRODUCTION 

For readers of this report, English as a Second Language (ESL) is 
a subject known well, but usually only because of experience as 
teachers or program planners. Few of us have learned English as our 
second language, or had to learn any language under the circumstances 
currently confronting refugees in Minnesota- This makes the efforts 
of judging the effectiveness of ESL efforts even more elusive* 

Yet effectiveness is important to people at all levels. This 
report was commissioned by the Minnesota Department of Education 
Office of Community and Adult Basic Education to evaluate the 18 
adult refugee ESL programs funded in 1985-86 through the Minnesota 
Refugee Program Office of the Department of Human Services, 

The purpose of the evaluation was to identify effective practices 
and make recommendations for future decision making and planning. 
There were two sources of information which we could use to explore 
this question. One was the data system used by the Refugee Program 
Office. The second was all of the people delivering and receiving 
services. We originally designed the project to rely on both. 
However, in the course of our work we discovered the following: 

1, No outcome data which might measure student progress would 

be available within the time frame of this inquiry. Even if such 
data were available, it would not explain why some students were 
progressing more rapidly than others. 

2. The current data system provides no information on employment 
outcomes for students . 

3, Teachers were often unable to articulate during interviews 
why certain strategies or methods were used, or what principles 
formed the basis of those choices, 

4. Students, while surprisingly open, were able to discuss many 
likes and dislikes, but were not able to address the question of 
what methods were most effective. 
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5. There is significant tension between the federally mandated 
employment focus and the educators' belief in the importance of 
literacy skills. This debate is at the heart of virtually all 
design and method decisions. * 

6. Current system design problems appear to be more significant 
than method selection in determining the effectiveness of ESL 
instruction. 



Because of these factors, we altered our project to rely heavily 
on perceptions of those in the field: administrators, coordinators, 
teachers and students. We focused our inquiry. on identifying methods 
and contexts, or program design variables, which those interviewed 
believed to be most effective. 

We were unable to match their perceptions with "hard data 11 on 
student progress, since such data was unavailable. Yet their 
comments are rich with lessons for practitioners, gleaned both from 
what they said and what they were unable to say. The first section of 
this report is our effort at reflecting their collective thoughts, 
pointing out differences of opinion, and drawing conclusions about 
effectiveness. 

In the six month course of our inquiry, we found ourselves 
repeatedly confronted with system -wide issues as similar themes 
emerged during interviews and were corroberated by observations in 
the field. For example, planners and decision makers expressed deep 
concerns over the current environment within which the Refugee 
Program Office operates: significant' federal reductions in funding 
are anticipated, reduced time eligibility for certain services is 
mandated and the federal emphasis continues to be on employment, 
which is at apparent odds with much of what teachers view as 
important in teaching English. 

These environmental factors make the future of ESL quite 
uncertain, regardless of what classroom methods are employed. 
Although the purpose of this project is to define those methods which 
are most effective, we have also tried to address the system issues 
which are so important to the future direction of these projects. 
These system issues are discussed in the section entitled Critical 
Issues for Public Policy Decisions. 
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B. METHODOLOGY 



1- PROJECT DESIGN PHILOSOPHY 



Learning English is the key for refugees in building a new life 
in this country. This assumption is not new, but there are several 
circumstances that make the problems of current refugees different 
from those of past immigrant groups. 

The first is the unprecedented rate of arrival of Southeast Asian 
refugees. From 1979 to 1985 close to 30,000 refugees settled in 
Minnesota, 85 percent of whom are Vietnamese, Hmong, Lao or 
Cambodian. (Other refugees arrive primarily from Ethopia, Roumania 
and Poland.) Second is the relative lack of marketable skills of the 
Vietnamese, Hmong, Laotian and Cambodian, especially those who 
arrived later in the U, S. . Third is the great difference between the 
cultural background of refugees and the cultural expectations in the 
0. S. 

Each of these problems has had a direct influence on the programs 
that try to teach English to refugees. For example, the high arrival 
rate produced an immediate high demand for language classes, but did 
not allow for an orderly planning and design of those efforts. The 
lack of marketable skills and the relatively low educational level of 
later arrivals made educators realize that traditional approaches to 
education may net work, and that many refugees needed more time than 
originally expected to learn the new language. The cultural 
differences influenced ESL classes by making teachers" realise that 
some in-cla3s methods ere not accepted or effective with some 
Vietnamese, Hmong, Laotian and Cambodian, and that methods should be 
adjusted according to the specific refugee group. 

In the past few years ESL classes have done well in recognizing 
these changes and have begun to incorporate them into their teaching 
efforts. It is important to note, however, that these efforts are 
still in their developmental stages. Although educators have begun to 
more clearly define the goals of ESL services and the problems 
involved in delivering those services, there is still substantial 
debate and experimentation about the specific methods to use. 

Recognizing that service delivery is still in the developmental 
stages is an important foundation for evaluation and project design 
efforts in ESL projects. This project is designed to provide the 
information which can assist in efforts to identify effective methods 
and can be used in defining future service directions. 
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This formative direction is especially important when one 
considers tho variety of ESL service delivery organizations involved. 
In a few cases they are well-established agencies that have many 
years of experience in community education. They are part of the 
education network and have good access to the innovative ideas that 
may helj them improve their ESL services- There are many other 
agencies, however, that are operating ESL classes because of their 
dedication to the resettlement of refugees and see language 
training as an important component, 

There is even more diversity among the delivery organizations 
when one considers which 18 agencies are under contract with the 
State. One agency is a private organization', three are Mutual 
Assistance Associations (HAA's)^ one trains volunteers for 
personalized tutoring, some are affiliated ^ith local churches and 
others operate within the Adult Basic and Continuing Education 
settings (ABCE). (Appendix A includes a listing of these agencies 
with summary information of who they serve, ) 

Having such a variety of service vendors is both a challenge and 
an asset. Although communication lines are not well established , 
there is an opportunity for agencies to learn from each other, with 
traditional educational agencies learning from the refugee agencies 
how to refine their methods, and the refugee agencies learning from 
the traditional agencies how to operate in this complicated human 
service sector. One role of this project is to enhance the transfer 
of such learning and to stimulate healthy project development. 

This situation creates an especially difficult task for the 
evaluators, because providing technical assistance is not the primary 
purpose of such an evaluation project. It is possible, however, to 
define the tasks of the evaluator in such a way as to provide the 
information necessary for public policy and planning decisions at the 
State and still provide directions and recommendations that can be 
used by local service agencies in their efforts to improve their 
service delivery. This is what we have attempted tp do in this 
project. 
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2. PROJECT ACTIVITIES 



This project was dependent on the participation of many teachers, 
project staff and students- They were the source of the information 
from which theses observations arose. Below are listed the activities 
of this project. They are included here to describe our efforts in 
involving these people in this formative process. 

Major components of this design were: 

1. Six design meetings with the Adult Refugee Coordinator at 
the Department of Education in order to refine the project 
design, review the history and current status of ESL in 
Minnesota and discuss major issues and debates in ESL 
teaching strategies and methods. 

2. Review of project files including the 1985-86 Request for 
Proposals, individual contracts, supplementary information 
on each of the sites, and pertinent literature. 

3. A preliminary meeting with project coordinators at which the 
project was introduced and input solicited for refinement of 
design. 

4. Three structured meetings with representatives of the State 
Refugee Program Office and the Department of Education, 

5. Site visits to 16 of the 18 locations where ESL services are 
provided. These visits included interviews with each program 
coordinator and with groups of teachers. About 50 teachers 
were involved, 

6. Five student focus groups, in which refugees discussed their 
opinions on the structure, of ESL classes, methods used, and 
their needs. About 50 students were involved. 

7. One focus group meeting with 8 teachers in which 
educational philosophies, methods and innovative 
ideas were discussed. 

8. A review of demographic and program data available through 
the Refugee Program Office. 



It is apparent from this list of activities that a great number 
of individuals directly involved with ESL participated in this 
project. This report is an attempt to reflect their perceptions on 
"what strategies and methods are most effective in teaching English to 
adult refugees. 
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3. CLUSTERING BY STUDENT ABILITY 



If one wishes to understand how different methods influence the 
rate at which students learn English, then one must take into account 
the other aspects which may influence this rate of learning. We 
considered the three mo3t important aspects to be: 

1. Student ability, which is affected by inherent intelligence, 
past general educational experience, past English education 
experience, cultural adjustment factors, and individual factors 
which may include age, sex, family norms and motivation. 

2. Program design factors, which include the times classes are 
offered, support services such as transportation and day care, 
links to other agencies, the role of coordinators, and size of 
classes. We have labeled these "Context Variables*'. 

3. Teaching strategies and methods, which include classroom 
organization, planning, materials and personal style. 

Given the importance of these aspects, a three-dimensional matrix 
would be most helpful in understanding which methods would have their 
most impact on which students, and in which settings. We have 
represented this way of thinking by organizing our findings into 
groupings of students, or "clusters". With this method we can control 
for student ability level and it becomes possible to discuss program 
design factors and teaching methods in roughly comparable 
circumstances . 

Our clustering system is based on the Student Performance Levels 
(SPL's) used for placement. We believe this is the best available 
proxy for the individual differences in learning abilities. Although 
it is a secondary measure, one can assume that students in similar 
SPL's have roughly similar individual factors that relate to their 
ability to learn English. 

The seven SPL's have been collapsed into three separate 
clusters. Cluster A includes SPL's 0, 1, and 2. Cluster B includes 
SPL's 3 and 4, and Cluster C includes SPL's 5 and 6. Both context 
and method observations are discussed within this clustering system. 
Care should be taken to remember that individual student differences 
may still have strong impact on the performance of a student in any 
given teaching situation. 

The following section identifies the context and method factors 
that appear to be most important in teaching adult refugees. They 
represent the opinions of a majority of those interviewed in 
combination with our classroom observations. The factors are 
organized in four sections, starting with those context and method 
issues that appear to relate to all Student Performance Levels, 
followed by sections dedicated to each of three SPL clusters. 
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II. CONTEXT A&JD METHOD VARIABLES 

ORGAN X ZED BY 
THE LEVEL OF STUDENT ABILITY 



This section summarizes the context and method variables that are 
perceived to be most effective in teaching English to Adult refugees. 
These perceptions were collected through visits to 16 of the 18 
programs in Minnesota, interviews with over 50 teachers, interviews 
with 16 coordinators/lead teachers, classroom observation at all site 
visits and focus groups involving over 50 adult refugee students. 

The context, strategies and methods variables listed here are those 
issues receiving the most emphasis by the people participating in 
these interviews and reflect the consensus opinions expressed to us. 
They are organized in priority order in each section with those 
issues that appeared strongest across all interview sessions being 
listed first. 

Please note that examples are included on many of these 
variables. These examples are to display specific activities that 
we observed or ideas brought up on our site visit interviews. They 
are not intended as a thorough list of recommended activities but as 
a means of clarifying the variables and providing some stimulating 
ideas m 
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A. CONTEXT AND METHOD VARIABLES FOR ALL STUDENT PERFORMANCE LEVELS 



CONTEXT VARIABLES 

1. Hire and Support Excellent Teachers 

Excellent teaching is the one variable around which everything 
else revolves. Excellence is comprised of several factors. In 
addition to teaching experience and training, excellence also 
means: commitment and professionalism, (taking the job vei-y 
seriously); interest in learning, (from students, other teachers 
and supervisors); flexibility to adapt to changing needs of the 
class or individuals; and concern and caring for students, 
including respect and personal interest. 

Examples: 

Create new materials or actively search for better 
ones. 

Take advantage of training opportunities. 

Use staff meetings to raise concerns and discuss 
methods . 

Prepare thorough lesson plans, but be able to shift 
from them as needed . 

Become aware of students' personal successes and 
problems, including those related to employment. 



No one set of experiences and training creates ideally prepared 
teachers. Yet the huge variety of observed backgrounds and system 
supports has yielded vast differences in classroom 
effectiveness. In addition, minimal funding levels have 
discouraged expenditures on preparation time or staff meetings, 
programs give teachers little opportunity to observe each other 
or share ideas, and the lack of long-term contracts creates an 
atmosphere of instability.- 

Hiring excellent teachers should be the first priority of the ESL 
program. In addition, increased emphasis should be placed on 
teacher training, workshops, and opportunities to exchange ideas 
on methods. It would be useful to establish a Minnesota 
clearinghouse and forum for materials sharing, since so many 
teachers spend so much time searching for or creating their own 
materials. 

8 
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ALL SPL'S 



Recruit Program Coordinators With Leadership Skills, and 
Strengthen the Coordinator's Role 

The best program coordinators we observed were knowledgeable in 
the field and committed to staff development. By contrast, at 
some sites teachers operated quite independently with no evident 
support or feedback from coordinators. 

Examples : 

Provide in-service training on ESL or assure teachers 
have access to it. 

Maintain contact with other ESL providers and awareness 
of what services are offered where. 

Provide periodic classroom observations; encoura?** 
teachers to observe each other. 

Interpret state policies for teachers, helping 
incorporate them into methods delivery. 

Provide individual consultation with teachers on 
teaching style, methods and materials. 

Use a defined evaluation procedure which provide feed- 
back to teachers. 

Search for and introduce new materials. 

Because one of the major impediments to effective leadership is 
time commitments elsewhere, either as a teacher or administrator 
of other programs, it would be well to hire coordinators to do 
only that job, and to work with more than one site if necessary. 
Strengthening the coordinator's role as illustrated through the 
above examples would not only provide supports necessary to 
improve the quality of teaching, but would contribute to the 
consistency of service delivery across programs. 

Promote Coordination With Other Agencies 

Coordination needs to occur in three directions: with other 
services on the educational continuum, with employment services 
and with resettlement services. The coordinating role should be 
centered in the program coordinator. While this may seem self- 
evident, it is not what is happening in the field. Teachers too 
often serve as counselors for their students. It should be noted 
that maintaining an active interest in students' lives, and using 
immediate problems as a springboard for classroom lessons are 
important teaching approaches. That is not the same as case 
management. The best programs have links to other agencies which 
effectively help refugees, leaving teachers free to teach. 

9 
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ALL SPL'S 



Examples : 

Maintain a working knowledge of other resettlement 
services in the community, and provide students 
with information and referral. 

Establish close working relationship with employment 
services; provide in-service training to teachers 
to maintain a realistic view of job markets. 

Maintain knowledge of requirements of other 

educational programs, in order to help students 
plan for these opportunities. 

Make appropriate cross referrals to other ESL sites. 



Structure Programs That Can Adjust to Students' Individual 
Educational Needs 

This is a variable closely related to teacher style, but is also 
a program design issue. It links ESL teaching to the philosophy 
of individualizing teaching to meet students' needs, and is seen 
as important because of its ability to speed the pace of learning 
within the framework of federally defined time eligibility. 

Examples: 

Provide open enrollment when appropriate for students 
who are unable to consistently attend. 

Allow teachers to move students to a different level 
in the middle of the quarter. 

Use classroom aides and volunteers to increase 

individual and specialized instruction time. 

Use in-home tutors to reinforce classroom learning. 

Maximize Instructional Time and Pace 

This variable, like the one above, is critical because of the 
time eligibility factor. According to students and teachers, 
holding classes daily helps retention, and the preference was for 
3-4 hours per day. The most effectively run classes had from 8- 
10 students, or no more than 20 students plus an available 
classroom aide. Students expressed frustration over the slow 
pace pursued in large classes, and in the loss of equal 
participation by all students. 
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ALL SPL'S 

Examples: 

Start classes on time and minimize breaks. 

Design lesson plans which enable the student to move 
smoothly from activity to activitj*. 

Minimize student talking and other noise distractions. 

Use class time infrequently for outside speakers whose 
presence does not contribute to language learning. 

Use volunteers to work with students individually. 

Promote English speaking on breaks. 
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STRATEGIES AND METHODS FOR ALL SPL LEVELS 



ALL SPL'S 



1. Teach Students to Learn 

This is a strategy much discussed and difficult to implement, but 
essential to the long-term success of refugees, both in education 
and in their other pursuits. In the classroom it is evidenced by 
encouraging questions and risk-taking. The liveliness of 
students who work with such teachers contrasts sharply with the 
passivity, often mistaken for respect, of students who rely on 
teachers for direction. 

Examples : 

Develop study skills, such as memorization, active 

listening, note taking and review of written materials. 

Teach students to use everyday references, such as the 
yellow pages, want ads, bus schedules and naps. 

Practice dictionary skills. 

Encourage questioning in the class. 

Ask students to solve problems, using English, rather 
than answering all questions immediately. 

2. Use Relevant Issues as Tools for Teaching 

The best teachers were attentive to this principle of relevance - 
adapting lessons to real situations present in student's everyday 
lives. This frequently involved adapting ready-made materials 
and/or creating materials geared to students' interests and 
concerns . 

Examples : 

Have students write or talk about their homeland, their 
journey to the U. S. , or their first experience in 
Minnesota* 

Discuss potential employment opportunities, using 

actual, current want ads; offer a field trip to 
sites where refugees have been or are being hired. 
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ALL SPL'S 



3. Use Flexible Responses to Immediate Issues 

While normally stated in the same phrase, relevance and immediacy 
are different elements of excellent teaching. The latter refers 
to the ability to flexibly respond to immediate circumstances or 
issues in students' lives. There are two dimensions to this. 
One is taking a problem-solving role, so that students are able 
to attend to lessons without distraction. However, this comes 
close to the case management role described above and is a 
m potential trap for teachers. The more important dimension is to 
turn immediate events or issues into English lessons. It is 
important to remember that there is typically conflict between 
providing sequence and order, and flexibly resiponding to 
immediate issues. 

Examples: 

Use weekend activities as the basis for classroom 

conversation, and preparation of a class news- 
letter to student vocabulary. - 

Use a student's need to take public transportation to 

the hospital that afternoon as the springboard for a 
lesson on fonas of transportation, map and schedule 
reading. 

4- Promote, Encourage and Enhance Learning Outside the Classroom 



This strategy is an important way to increase the amount of 
practice students can get, speed their learning and aid in 
transfer of knowledge to new situations. Students who were 
interviewed were highly interested in opportunities to practice 
English, but virtually all reported that they rarely speak 
English outside of class. The few who were employed reported 
difficulty in using English on the job. 

Examples: 

Assign homework daily. Provide consumable materials. 

Use cassette tapes for students to practice listening 
and speaking. 

Structure dictionary lessons that can be done 
independently. 

Promote students' listening to news, writing down new 
vocabulary words and discussing them. 
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ALL SPL'S 



5, Involve Students in Educational Goala 

The methods for implementing this strategy vary considerably from 
the lowest to highest level of English ability. But the 
principle is consistent; students who were helped to understand 
and articulate educational goals seemed more actively motivated 
to learn. Some teachers explained that it is hard to convince 
some students of what they need to learn, and slow them down to 
learn things thoroughly. 



Examples: 

At the lowest levels, maintain an order and simplicity 
that in itself gives students a sense of progress 
and accomplishment. 

Discuss with students the lesson plan for the week, and 
review progress at week's end. 

Set goals for the classroom. 

Set individual learning objectives. 

Maintain a record sheet for homework as well as class- 
room activity, and use it to show progress to 
students . 

Help students think about and articulate the link 

between English and its application in their lives. 

Recognize achievements of students publicly. 



6* Balance the Use of Appropriate Materials 

Students and teachers both voiced the opinion that the use of a 
variety of methods, with corresponding materials, was the route 
to faster le&rning. While this may seem basic, many teachers 
seemed to have difficulty with this balance, making a particular 
set of materials the focus rather than the aid to learning. A 
dearth of appropriate materials was a frequently cited problem, 
but this may indicate system isolation and funding problems 
rather than a real lack of materials. Materials should be 
appropriate to adults and should balance the relevant experience 
of the refugees with cultural orientation to the U.S. 

Examples: 

Use a brief conversation in a variety of ways: copy, 

memorize, recite orally with a partner, dictate and 
write, pull key vocabulary words, work on grammar. 
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ALL SPL'S 



Use spelling and vocabulary lessons to reinforce writing and 
reading materials. 

Organize field trips to employment sites so students can see 
physical layout, sense the work atmosphere and perhaps 
meet potential employers. 

Build practice conversations based on the actual site visit. 
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B* CONTEXT AND METHOD VARIABLES FOR STUDENT PERFORMANCE LEVELS 0-1-2 



CLUSTER A 

CONTEXT VARIABLES 

1. Attract new refugees who have never attended ESL classes in the 
agency's community. 

There is some concern that there is a lack of adequate 
advertising and recruitment for ESL students. Many agencies seem 
to depend on the word of mouth system from past and present 
students and informal contact with resettlement agencies. These 
methods may not be reaching all potential students and may become 
lesis effective as local attention to resettlement decreases. 

Examples : 

Require less rigid regulations for enrollment, 
(e.g. open enrollment) 

Use a culturally comfortabl setting (building and 
classroom) . 

Arrange transportation to and from class. 

Advertise to reach potential students. 

Actively coordinate with sponsoring and resettlement 
agencies. 



2* Maintain students' attendance once they have begun attending ESL 
classes. 

Examples : 

Arrange support services, 
transportation 
child care 

ability to refer/address immediate crisis 
resettlement problems 

Have translation services available, 
bilingual staff 

quick access to other bilingual resource 
( MAA , student , caseworker ) 
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SPL 0-1-2 



Organize classes of lower performance level students with as 
similar English ability levels as possible. 

Mixing new lower level students with other students who have 
higher English proficiency and who have been in classes for a 
while does not seem to be a good idea. It isolates the new 
students and increases their levels of self doubt* It also 
reduces the chance to address the unique immediate needs of these 
new students, such as their resettlement problems and lack of 
study habits. 

Organising by student performance level should also make it 
easier to select materials and develop methods and allow for more 
appropriate group work, especially for the lower levels (See 
method issues discussed below. ) 



Treat students as adults 

This context issue is one of substantial debate and there is much 
variance across classrooms about how students are treated, 
especially in the lower level classes. We have identified this as 
a key context variable because of its importance in establishing 
an atmosphere of learning in which future context and method 
goals can be addressed. Of special note are the 
interrelationships of this variable with "Teaching to Learn", 
"Attention to Individual Needs" and "Ability to Maintain 
Attendance". 

Examples: 

Use materials appropriate to the activities of an 
adult 

Use a class process that recognizes and uses adult 

students' abilities to reason, to think abstractly 
and to problem solve. 

Keep adult students informed about the reason for 
selection of teaching methods and materials. 

Involve students in defining personal goals for 
theix* participation in class and use this 
information to decide on classroom design within 
the existing curricula. 
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SPL 0-1-2 

Address student expectations in classroom design and process. 

Some students have a preconceived notion about how schools should 
operate with an emphasis on a formal classroom process and 
written materials. Even though such a process is not necessarily 
the most effective, it is important to address these 
expectations, especially in the lower level classes, If these 
newer students are surprised or disappointed too much, they will 
just stop coming. 

This creates a dilemma for some programs that wish to design 
classes around informal methods, but such styles should be 
introduced as students become accustomed to the class and can 
begin discussing the reasons behind the new methods. 

As classes progress, student expectations should continue to 
influence classroom design, but should be incorporated in the 
active process during class of discussing individual $;oals, 
classroom activities, and student progress. 



Examples : 

Use consumable materials 

Use books and organise materials that enhance students sense 
of ownership and promote a sense of progress. 

Use written materials for homework 

Use teacher-directed methods in the very early days 

Promote the discussion of personal educational goals 
and expectations. 
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STRATEGIES AND METHODS FOR SPL 0-1-2 



SPL 0-1-2 



2L Actively involve atudenta in the clasaroom activities* 

Note the relationship of this issue to the issues of "Treat as 
Adults", "Teaching to be Students" and "Teaching to Learn'* . 

Examples: 

Incorporate physical activities and experiential techniques 
such as Total Physical Response* 

Promote active listening. 

2* Help students solve their immediate crisis resettlement problems 
that hinder their ability to learn in class * 

Examples: 

Encourage students to identify problems they need help 
with in order to attend and learn. 

Maintain an atmosphere in which students are willing to 
ask for help. 

Address such problems through teacher or other 

staff capacity, or through active referral process 
to local caseworker. 

3. Focus on students gaining a familiarity with the English sound 
system and a basic proficiency in English sentence structure. 

Examples: 

Focus on oral skills, listening and repeating. 

Use group activities that require active listening and 

repetition, incorporate immediate feedback and promote 
the willingness to try speaking skills in class. 
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SPL 0-1-2 



Teach study habits 

It is generally believed that many of the lower level sturlents 
who have had little or no educational experience are also 
unprepared and inexperienced in learning through study. Those 
students without these study skills will not learn as fast as 
others and will fall behind. This places the responsibility of 
teaching these skills and establishing study habits on the 
teachers of these introductory levels. 

Examples: 

Practice and teach memorization. 

Practice and teach active listening 

Use written materials to study on their own. 

Emphasize the importance of reviewing past information 
to enhance remembering. 

Promote the practice of speaking English outside the 
class. 
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C* CONTEXT AND METHOD VARIABLES FOR STUDENT PERFORMANCE LEVELS 3-4 



CLUSTER B 



CONTEXT VARIABLES 

The context and method variables which apply to Student Performance 
Levels 3 and 4 tend to be variables which are important in their 
basic form in the lower performance levels and evolve into the 
methods and context variable for the highest levels. Because of this, 
many of the variables included in this section use relative terms 
such as "increase", "decrease" or "emphasize". Our intent is to 
identify these key variables and to provide some direction on tiie 
rate of transition as these methods and contexts evolve through the 
SPL clusters. 



1. Design classes so that students can begin receiving more 

individual attention as their individual differences begin to 
surface. 

Examples: 

Enhance and promote students voicing their opinions and 
interests. . 

Use student interests and educational goals as one 
basis for deciding on class content. 

Provide the resources (materials and personnel) to allow 
for more small group activities, 

2. Promote the active involvement of students in the design of their 

classes. 

As students become tamiliar with our educational setting and 
progress to a level where they can communicate within the class, 
greater and greater emphasis should be p?aced on student 
responsibility for their educational goals and student 
involvement in the design and control of their own education in 
the classroom. SPL 3 and 4 are not too soon to start. 

Although it sounds like a grand platitude, those programs that do 
expect such involvement seem to succeed in assisting their 
students in becoming clear about their goals, and assertive and 
motivated about their education. 
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SPL 3-4 



Examples : 

Promote teachers * role in explaining why certain lesson 
plans and classroom activities have been selected. 

Promote student direction for the selection of future 
classroom activities. 

Design and use student evaluation of project design, teacher 
performance, materials and classroom activities. 

3. Increase the involvement of outside resources on site, 

especially employment agencies, prospective employers and 
volunteers that can help create realistic classroom experiences. 

As a general rule, the students at level 3 and 4 have had 
sufficient time to adjust to the immediate problems of 
resettlement and are able to converse at a basic level with 
strangers. They are, however, often unwilling to seek such 
interaction on their own so ESL programs should arrange 
oppoptunities for students to converse with other people often. 

The goal should be to incorporate the use of outside people 
within the context of the class design and should be controlled 
by the teacher, but the frequency of using these outside 
resources should increase as quickly as possible. 

Examples : 

Arrange for volunteers to assist in the classroom as 
aids for an extended period of time. 

Arrange for community representatives who have 

knowledge in an area of special interest to visit 
for one or two class periods to discuss their 
area. (e.g. police, lawyers, job service, big 
employers , nutritionists ) 

Plan after class or break time "teas" where volunteer 

citizens can meet and converse in English with the 
students. 

I. Identify local community activities where students could practice 
their English speaking skills and actively promote student 
participation in those activities. 

Examples : 

Coordinate with local community centers, churches and 
other agencies that organize informal social 
gatherings and promote events in which students 
can participate. 

Promote the use of ESL class or building space by other 
agencies so that students can participate in 
familiar space. 
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SPL 3-4 

STRATEGIES AND METHODS FOR SPL 3-4 



1 % Increase activities that integrate reading and writing into the 
teaching plan in an effort to construct the foundation of 
literacy training* 

There is substantial agreement by many that basic literacy is a 
necessity for refugees, especially if they are interested in 
finding work that has the potential for future improvement. 
Students are almost unanimous in their request for more reading 
and writing skills even after their speaking skills are very 
refined. 

The issue for ESL classes seems to be when to attend to these 
basics and how much attention should be given to them within the 
present employment context. In general, most agree that starting 
these basics early is the best, and there should be a consistent 
effort across all classes to build the basic foundation of 
■ literacy. 

Examples: 

Begin using written materials as the source of topics 
for conversations in class. 

Increase the expectations that students will read 
materials in preparation for class. 

Have students maintain a written notebook of notes and 
lessons written in class and at home. 

Begin introducing lessons that require students to 
communicate in writing. 

2* Introduce more spontaneous conversation into the class process. 

As students develop the ability to converse, the use of 
spontaneous conversation in the classroom becomes easier. Such 
spontaneity not only increases student involvement but also 
assures that the topics discussed are more relevant and immediate 
for the students. 

Examples : 

Begin offering opportunities for students to bring up 
topics of interest in the class and adjust the 
lesson plan to incorporate them. 

Place greater expectations on students to introduce 

ideas or topics that they would like to discuss. 
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SPL 3-4 



Use current event topics as the basis of lessons. 

Assign homework that includes relevant topics that can 
be discussed in the next class periods. 



3. Continue to reduce the amount of teacher-directed activities and 
increase the amount of student-directed activities. 

This variable is interrelated to many other variables identified 
as being important, including "Treating Students as Adults 0 , 
"Increasing "Student Participation", and the general goal of 
assisting students in becoming more involved and assertive. 

Examples: 

Reduce the authority position of the teacher. 

Begin introducing sections of the class in which 
activities are selected by the students. 

Use more small group activities. 



4. Begin increasing emphasis on teaching students to solve problems 
on their own. 

Examples : 

Answer fewer questions posed to the teacher and 

redirect them to the students for a solution. 

Reduce the position of authority of the teacher 

(by changing teacher/student positions, means of 
addressing each other, etc.) 

Request direction or advice from the students on 
process or topic issues. 



5. Structure classroom activities to provide more and more individual 
attention as the need arises. 

Examples: 

Use smaller groups that are organized by interest areas 
or activities. 

Use aids or volunteers to give individual attention to 
students who have specific interests or specific 
educational needs and desires. 

Begin introducing student-to- student teaching in small 
groups • 
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D. CONTEXT AND METHOD VARIABLES FOR STUDENT PERFORMANCE LEVELS 5-6 

CLUSTER C 



CONTEXT VARIABLES 



1. Design classes to enhance the ability to work individually with 
students as much as possible. 

Examples: 

Group and regroup students periodically by 
performance level and interest. 

Keep the classes small to moderate size (6 to 12) 
Coordinate the use of tutors and aids. 

Provide community companies/agencies/individuals for 

use in class exercises so long as they have a clear and 
explicit role in teaching the language skills for the 
day. 

2. Actively coordinate with other local education providers. 

Students leaving ESL and wanting more education should be 
assisted in this transfer. - Also, students who wish to study in 
another setting concurrent with their ESL enrollment should be 
actively assisted. 

Examples : 

Keep informed and maintain active contact with other 
educational program options. 

Actively inform students of these options. 



3. Coordinate, promote and refer individual students who want or 
need education outside of the ESL class. 

Examples : 

Provide materials for self-study for those students 
with the time and inclination. 
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SPL 5-6 



Arrange for individual tutoring before/after class for 

people who need remedial help and/or for people 

wishing to study more- 
Coordinate with other sources of classes or tutors and 

actively refer students who want/need the 

assistance. 



Develop and maintain the ability to address students' more 
immediate employment needs. 

Some will be in jobs while 'they are in ESL classes, and more will 
be actively seeking employment. 

Examples : 

Schedule classes to allow for working students to 
attend, 

Dse employment materials that apply to specific 
local employment options. 

Maintain active coordination with local employment 

services and possibly with some major employers. 

Keep teachers and students informed of local employment 
service options. 
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STRATEGIES AND METHODS 



SPL 5-6 



1* Ssphasiae accuracy of pronunciation, structure, vocabulary and 
writing. 

Students should already know the basics of English sentence 
structure and should be able to converse enough to be understood. 
If this is true, then the upper levels of ESL should include 
methods and activities that allow students to improve their 
pronunciation and their ability to use the English language 
accurately in speaking and writing. 

This emphasis should be balanced with the continuing need to 
review the basic skills, to expand vocabulary and to assist 
individuals with their specific needs. 



Examples : 

Correct students as they make errors in speaking during 
lessens. 

Provide correction and immediate personal feedback on 
lessons and homework. 

Emphasize 1:1 teacher/student or student/student 
interaction. 



2* Integrate reading , speaking and writing. 

As students progress in their speaking and basic English 
proficiency, teaching methods should be able to integrate 
reading, writing and speaking so that students learn to use all 
of these as interchangable methods of communication. 

Examples : 

Use written materials as the source of topics for 
class discussions. 

Expect students to read material in preparation for 
claas activities. 

Teach note-taking skills and promote writing down 

information and instructions during class in order 
to assist their individual studies. 
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SPL 5-6 

3* Accommodate individual differences in the pace of learning in the 
classroom. 



4. Promote and use discussions within the olassroom that require 
students to converse 
memorized materials. 

This issue is interrelated to the issues of "Treating Students as 
Adults", "Relevance" and "Immediacy" . 

Examples : 

Promote student-generated comments/questions during 
class* 

Instantly revise the 'planned' lesson to take 

advantage of an opportunity for spontaneous 
discussion. 

5. Take advantage of students' abilities to be independent, self- 

directed and to learn on their own. 



i 
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Examples ; 

Use small groups or individual study. £ 
Promote student -student interaction. 
Use the teacher as facilitator with small groups. 
Use aids, volunteers, outside participants. 
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Examples : 

Promote student-directed activities in the classroom. £ 

Offer ideas and materials for independent study in ^ 
the class and outside the class. M 

i 
i 
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E. SUMMARY OF STUDENT PERSPECTIVES 



As part of this project five focus groi ^ with present ESL 
students were organized. Two groups met wita as at Pratt school in 
Minneapolis, one in Eagan and two in Rochester. There were two 
sessions each for students in class levels 3-4 and 5-6 and one with 
students in level 1-2. Our goal was to seek the opinions of students 
about the most effective methods for learning English and to compare 
their perceptions to those of the teachers and other program staff. 

The sessions were generally informal, involved 8 to 15 students 
in each group and lasted between 60 and 90 minutes. The students were 
informed of the sessions during their class times and were asked if 
they wished to participate. The working outline used for these focus 
groups i-s included in Appendix B section 3. 

The issues brought up by the students were incorporated to a 
large degree into the method, and context summaries in this report, 
but there were certain student perceptions and some observations 
about the focus groups that should be highlighted separately. This 
section lists those issues that we believe to be of importance in 
future planning. 



OBSERVATIONS 



Students talked in terms of like/dislike and were generally 
unable to define methods as effective/ineffective. 

Students were more willing to state opinions and criticisms than 
we were led to believe by teachers and others. 

Individual differences were evident in all focus groups but 
became more defined in upper level groups. 

4 

Almost no opinions were stated about English lessons received 
elsewhere or any comparisons to other education. 

Students were generally unclear about how long it would take to 
learn English. 

Groups were often dominated by 2-4 individuals with 1-3 others 
saying almost nothing. 

It was easier for students to respond to very specific questions 
than to generate ideas in response to open-ended questions. This 
was less true, however, for the highest SPL groups. 
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Answers to "why are you learning" covered a r^ide range. Some 
students wanted to be able to talk to American friends, to learn 
to drive or shop or to keep up with their children as they 
learned English. 

Finding employment was mentioned more frequently than other 
motivations but it was not necessarily the central reason for 
taking classes. 

Some students noted that their goal was to attain a job that 
had future potential, so they were frustrated about the 
minimum training offered through ESL, 



STUDENT COMMENTS AND PERCEPTIONS 



Almost all agreed to the importance of practicing speaking 
English outside the class but almost no one spoke English at home 
or in any other setting in which they were not forced to use 
English, 

Their common message was that they want to l^arn faster and want 
to pick up the pace of the classes. Their specific 
recommendations were: 

Don't waste time on class schedule. 
Start on time. 

Do not tolerate late comers. 
Minimize break times. 

Maximize use of class time. 

Don't address remedial problems at the expense of 
others . 

Maintain efficient class size. Ranges were generally 8 to 12 
students per teacher. 

Students want more of everything; vocabulary, pronunciation, 
reading, writing, conversation. 

Some were willing to sacrifice thoroughness and accuracy in order 
to go faster. 

Others were interested in going carefully so they could get it 
right and remember it. 
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In general, students liked the periodical reorganizations of 
classes according to progress and level of ability, but did not 
like the lack :>f continuity of materials when they change 
teachers . » 

They would prefer to work through a book and finish it instead of 
skipping from chapter to chapter. They mentioned their 
frustration when they sometimes repeat chapters or skip sections 
when changing teachers. 

They wanted to learn faster but also responded that they would be 
unable to study at home more, had few chances to practice and 
have little time to do more than they are doing now outside their 
family and work obligations. 
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III. CRITICAL ISSUES 
FOR PUBLIC POLICY DECISIONS 



During the many site visits and interviews in this project it 
became evident that there were a number of undercurrents and 
recurring issues that affected the basic ability of agencies to 
deliver quality ESL services . This section is our attempt to 
identify these "critical issues". We wish to help people recognize 
the importance of these issues, stimulate the necessary discussion 
and make sure that clear policy and project design decisions are made 
which address each of these critical issues. In our opinion such 
decisions are essential in the efforts to improve the ESL programs in 
Minnesota 
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A. SUMMARY OF CRITICAL ISSUES 



I. MANY AGENCIES HAVE BEEN REDUCED TO BELOW "CRITICAL MASS" SIZE 

Many agencies have been reduced in size to a level that does not 
support the dynamics necessary for a healthy organization and 
makes it very difficult to support a professional staff. 



Training and development time is very minimal. 

Teachers and programs are isolated from each other. 

Many teachers consider ESL teaching as part-time work and 
must maintain other jobs while they teach. 

Salaries and benefits are not competitive with other 
professional educators. 



Reduction in size of classes may be due to this 'critical mass 1 
concept rather than a reduction in demand. People may not be 
coming because they are unaware of the class, or the size or 
quality of class has been reduced. 



II . SOME COMMUNITIES HAVE ATTEMPTED TO FILL THE GAP 

In some communities local resources have been rallied to provide 
basic English instruction to adult refugees. 

Often these are not money resources, but in-kind time, space, 
administration, or support services. 

These resources are not very secure. It seems as though they are 
offered as emergency stop-gap measures in which the arrangements 
could change overnight. 

The availability of alternative .resources differs significantly 
among communities and the ability of local staff to secure 
alternative resources varies among agencies. 

There are significant differences between urban, suburban and 
small town settings. 

Dedication/knowledge about local resettlement seems to prepare 
people better for this task. 

Adult Basic and Continuing Education (ABCE) funds have been the most 
common resource for sustaining the waning ESL programs but the use of 
this resource varies significantly among agencies. 
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In some situations it is difficult if not impossible to 
differentiate between the ESL program and the ABCE program. 

This transition has gone so far in some cases that ESL is 
considered just another source of money to support specialized 
activities within ABGE . 

Some present policies have promoted this transition to ABCE. 

All money is used for instructional time, little for 
administration, none for training, preparation. 

The RFP and reporting process are viewed by many as cumbersome, 
especially for programs that receive smaller grants. 



III. THE NEW EMPHASIS ON EMPLOYMENT IN ESL HAS NOT SIGNIFICANTLY 
CHANGED THE CONTEXTS OR METHODS USED IN ESL INSTRUCTION. 

The employment emphasis is no great detriment to attempts to 
teach English literacy to refugees, but it also does not help 
much. There are more problems adapting in lower level classes. 

Agencies and teachers generally do not view their efforts as part of 
the employment service continuum. 

There is little or no use of employment as an outcome measure by 
teachers or agencies. 



IV. TEACHING LITERACY IS CONSIDERED ESSENTIAL BY MOST TEACHERS AND 
ADMINISTRATORS . 

Most agencies have attempted to continue teaching literacy even 
though the federal directives for employment focus seem to make 
reading and writing secondary. 

There is some dissatisfaction since the employment goals are short 
term where many of the goals for language and literacy training 
require long-term efforts. 

There seems to be little state policy direction on how to 
balance this dilemma, so local agencies adjust any way they 
can. 

Effectively teaching literacy requires special skill in the 
classroom, and past experience and training in teaching reading seems 
to be an advantage for teachers in ESL programs that emphasize 
literacy skills. 

Most agencies and teachers consider their efforts as part of the 
local continuum of education services. 
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V. POLICY DIRECTION IS UNCLEAR ABOUT THE NEED FOR PROFESSIONAL 
EDUCATORS TO TEACH ESL CLASSES 

There is a dilemma when deciding what skills are important for 
teaching ESL- It is rare to find individuals who are both 
knowledgable of the resettlement services for refugees and skilled in 
teaching English to adults. As a general rule, resettlement people 
have less skill and experience teaching, and experienced teachers 
know less about the resettlement efforts in the community. 

Most teachers of ESL classes are quite isolated from those 
systems that could teach them teaching techniques and specific 
ESL methods. 

Although the limited state and regional training seems good, it is 
not offered often enough. 

The training that is offered assumes that all teachers have the basic 
skills and abilities about general teaching techniques. However, 
some teachers are in need of assistance in this area and many are 
interested in improving their skills. 

Reduction in ESL monies has made it less likely to support 
professional educators as ESL teachers. 

Present working conditions are often part time with limited 
benefits, minimal preparation time and relative isolation from 
other teachers. 

Individual methods of instruction have evolved to be quite 
different among classrooms and among agencies because of 
both the isolation of some programs and the emphasis on teacher 
autonomy in design and method selection. 



VI. THE PRESENT DECENTRALIZED DELIVERY SYSTEM DOES CREATE SOME 
PROBLEMS 

Although decentralisation addresses a concern for refugee 
access, the trade-off may be reduced quality and an overall 
program that is not unified. 

Having many small sites adds to the "critical mass 1 ' problem 
discussed above. 

One result is the mixing of student ability levels in one class 
which is very difficult for teachers to manage , 

Scattered sites add to the wide variance in the kind and quality 
of services provided. 
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Providing leadership, support and training by the state is more 
difficult with a decentralised approach. 

There were frequent comments on the need for more 
training* 

Isolated teachers find it more difficult to share ideas 
and gain support from other teacners. 



VII. MOST TEACHERS FEEL UNSUPPORTED AND ARE ISOLATED IN THEIR 

ATTEMPTS TO DESIGN CURRICULA AND SELECT MATERIALS FOR THEIR 
CLASSES 

Many teachers are totally responsible for the development of 
their curriculum. 

Many receive little or no curriculum development assistance from 
their coordinators, especially in the smaller programs with one or 
two teachers and part-time coordinators. 

Few programs allow teachers time to dedicate to curriculum 
development so much of it is done on personal time. 

There is no capacity to share ideas and materials with teachers from 
other programs other than through occasional workshops and site 
visits. 



VIII. TEACHERS HAVE VERY FEW OPPORTUNITIES TO IMPROVE THEIR TEACHING 
SKILLS WHILE ON THE JOB. 

The organized training options are only occasional. The workshop 
sessions that do exist spend little time on specific teaching skills. 

Teachers are rarely observed and critiqued in their classes by either 
coordinators, fellow teachers or trainers. 

This lack of training adds the the isolation of teachers and promotes 
the development of significant differences across classes. 
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B. RECOMMENDATIONS FOR FUTURE ANALYSIS 



1. Student Performance Measurements 

This project has been a start at identifying methods that are 
most effective in teaching English. It is important to remember that 
these methods have been selected based on the perceptions of project 
coordinators, teachers and students interviewed throughout the 
project. Although it is constructive to learn the perceptions of the 
people who are closest to the classroom, it would be a worthwhile 
effort to begin collecting data that would allow measurement of the 
actual impact of given methods on students. If this is a direction 
desired for future data analysis, there are two elements which are 
necessary to develop. 

1. An accurate measurement of student performance within each 

class that can be applied across all programs. 

2. A data system that allows information on students to be 

collected and analyzed in order to judge which methods are 
more effective. 

Progress is being made on both of these elements with the 
application of the BEST test and the developments on the Employment 
Competencies Checklist. Each has its potential but care should be 
taken in deciding which tool or combination of tools is most likely 
to work most effectively. 

At the time of this study the BEST test had been introduced in 
the field only a few months earlier and the Checklist was just 
introduced. We have included some very subjective comments here on 
the BEST test and SPL Checklist to consider in future applications. 

1. There was substantial variation among agencies in the amount 

of attention on the application of the BEST test. 

2. There seems to be a need for training on the application of 

the test and on its use at the local level. 

3. Most teachers and coordinators thought the BEST test Core 

Section rated individuals with oral skills too high. Also, 
without the literacy section there is not a complete profile 
of students' reading and writing skills. 

4. Many use the BEST test for initial placement of 

students but also used interview or first-week teacher 
observations as the basis for final placement. 

5. The one-on-one testing procedures are very time consuming and 

introduce significant variance of results due to 
individual testing practices. 
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6. Observations about the Checklist were limited because 

many had not received the materials during this project 
period, but most were enthusiastic about the concept and 
thought this structured teacher opinion format had potential 
to be useful. 



2* Data System Recommendations 

The second key element in actually measuring the effectiveness of 
teaching methods is the development and use of a data system that can 
keep track of student performance and can be used for analysis. 

The present data system that is being developed does lend itself 
to use in comparing student performance once the data is collected 
for at least a year. There are, however, a number of considerations 
we recommend as this system continues development. 

1. The data system should develop a means of clustering students 

by performance level as recommended in this study, or by 
future clusters, as they are refined. 

2. Student performance levels should be displayed statistically 

to represent the variance within agencies or within 
clusters - 

3. Summary reports should be regularly and quickly supplied to 

the local agencies, and access to the data system should be 
available to local agencies for planning purposes. 

4. Turn around time for data should be reduced. 

It appears that such a data system could be used to track actual 
student performance and could be applied to the methods identified 
here to actually measure which methods are most effective in teaching 
English to adult refugees. 
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APPENDIX A. 
SUMMARY OF THE AGENCIES 



Included here is a listing of the agencies under contract to 
deliver ESL services and a summary of their goals for the year as 
defined in their contracts. The summary is intended to give the 
reader a sense of the diversity of these agencies and to assist 
program people in discovering who else is working under similar 
circumstances . 

Please note that after two quarters of operation many of these 
agencies have experienced a mix of students quite different from what 
they predicted in their contracts, so use these data only as a 
general guideline in describing either the sise, scope or diversity 
of the students population. 
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I-SL EVALUATION PROJECT 
Nelson and Bringewatt 

SUMMARY OF CONTRACT GOALS 



AGENCIES 



ANOKA-HENNEPIN 



STUDENTS 
TOTAL 7. 

30 27. 



ASSOC... HMONG WOMEN * 165 97. 



AUSTIN 



BURfJESyiLLE 



60 37. 



20 IX 



MN. CAMB. BUDDHIST SOC. * 60 37. 



DULUTH 



FARIBAULT 



INT'L INSTITUTE 



50 37. 



25 17. 



110 67. 



PRIORITY GROUPS 
PG1 PG2 PG3 



22 8 0 
737. 277. 07. 



0 0 165 
07. OX 1007. 



60 0 0 
1007. 07. 07. 



6 14 0 
307. 707. 07. 



4 32 24 
77. 537. 407. 



35 15 0 
707. 307. OX 



18 7 0 
727. 287. 07. 



33 39 48 
307. 35% 447. 



PERFORMANCE LEVELS. . TCHRS 



I 

1 
I 



L1L2L3L4L5L6 



2 6 6 7 4 5 
77. 207. 207. 237. 137. 177. 



100 65 0 0 0 0 
617. 39X 07. 07. 07. 07. 



2 10 29 7 8 4 
37. 177. 487. 127. 137. 77. 



0 0 3 10 4 3 
07. 07. 157. 50% 207. 157. 



60 0 0 0 0 0 
100X 07. OX OX OX OX 



15 20 9 1.1 2 5 1 
307. 407. 187. 22X 47. 107. 



10 10 3 2 

407. 407. 127. 87. 07. OX 



0 24 51 36 9 0 
OX 227. 467. 337. 87. 07. 



I 
I 

1 

I 
1 

1 

1 
i 
i 
I 



LAO FAMILY (BRING. MAA)* 60 37. 



5 5 50 40 20 0 0 0 0 

87. 87. 837. 677. 337. OX 07. 07. OX 



LAO FAMILY (EMPLOYMENT)* 120 67. 



0 20 100 
OX 177. 837. 



60 60 0 0 0 0 
507. 507. 07. 07. OX OX 



MINN. LITERACY COUNCIL ** 



ERIC 
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METRO DEAF SENIORS 



32 2X 



0 0 32- 16 16 0 0 0 0 1 

OX OX 100X 50X 50X OX OX OX OX 



HOORHEAO 60 3X 22 36 2 IS 9 12 10 10 4 

37X 60X 3X 25X 15X 20X 17X 17X 7X 



HPLS 559 29X 355 149 55 115 138 84 86 86 50 ' 7 

64X 27X 10X 21 X 25X 15X 15X 15X 9X 



ROBBINSDALE 55 3X 30 19 6 0 4 19 22 8 2 1 

55X 35X 11 X OX 7X 35X 40X 157 4X 

ROCHESTER 265 14X 240 15 0 50 50 45 40 40 30 6 

91X 6X OX 19X 19X 17X 15X 15X HX 

ST. CLOUD 33 2X 2 24 7 2 24 7 0 0 0 2 

6% 73X 21X 6X 73X 21X OX OX OX 

ST. PAUL 192 10X 192 0 0 24 48 52 40 24 4 8 

100X OX OX 13X 25X 27X 21X 13X 2X 

WESTONKA 40 2X 6 27 7 0 10 23 7 0 0 2 

15X 66X 1SX OX 25X 58X 1SX OX OX 



TOTALS 1936 100X ...1030 410 496 ...511 514 343 278 195 107 45 

53X 217. 26X 26X 27X 18X 14X 10X 6X 

* MAA Priority Groups 
are reorganized to 
match ESL PG's 

** MLC does not provide 
direct service. They 
train and coordinate 
volunteers. 
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APPEiiDIX B. 
OUTLINES FOR AGENCY SITE INTERVIEWS 



The interviews during this project were not formally structured 
with predetermined and prewritten questions. They were guided 
conversations designed to elicit peoples' perceptions about the 
effectiveness of methods and contexts for teaching English as a 
second language. 

These conversational interviews were guided with the aid of a 
working outline on order to assure that all relevent topics were 
covered during each interview. Those working outlines are included in 
this appendix so that those who are interested can review them. 
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1. INTERVIEW WITH PROJECT DIRECTOR 



With the person directly responsible for the administration and 
management of the ESL project. May in some cases be the lead teacher 
when the Director does not have day-to-day contact with the program. 

Estimated time for the interview 45 minutes 



1. (Introductions and quick explanation of the project and of the 
purpose for this site visit,) 

2. Quick review and confirmation of the description of the program 
setting as described in the agency file materials, including: 

reason for starting ESL classes locally 
years of operation 

si3e of program (students, staff, budget) 
relationship with parent organization, if any 
description of local refugee community 
refugee community served 
geographic setting 

experience of teachers ' 

experience of agency with refugees and with education activities 

3. Discussion of the present decision-making process for designing 
and administering ESL classes, including: 

original purposes for starting classes 
philosophy behind the present design of the class 
role of administrator in supervising teachers 
process used to decide major context and method issues 

4. Discussion of program design and evaluation efforts within the 
agency, including: 

monitoring of classroom effectiveness, including teachers, 
contexts , methods . 

process used to incorporate experiences in the classroom into 
revisions for future program improvements 
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5. Discussion of the role of the State in monitoring and in 
providing support, including: 

perception of quality and direction of State efforts 
time needed for monitoring efforts 

the returns to the local agency for these monitoring and 
evaluation efforts 

6. Discussion of present outcome measurements 

direction of new performance measures 
direction of the new data system 

7. Relationship with other local service agencies, including 

recruitment of students 

other educators to advise on class design 

agencies providing other services that refugee students may need 
other educational opportunities for students during or after ESL 
employment referral agencies 
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2. INTERVIEW WITH TEACHERS 



With 1 or 2 teachers in the small agencies, and 3 or 4 teachers in 
the large agencies. 

Estimated time for the interview, 90 minutes 

1. Description of present program, including: 

classroom size 
student/teacher ratio 
physical setting 
class schedule 

students being taught (ethnicity, age, level) 

2. Discussion of how methods/materials were chosen and why 

3. Discussion of which students are most successful and why 

4. Discussion of which methods work best for which students 

5. Discussion of the employment emphasis of ESL classes 

8, Discussion of relationship between administration and teachers 

decision-making and problem-solving processes 
level of support from the top 
staff development 

7. Discussion of present student performance measures 

BEST test 
SPL system 

ideas for better measurements 
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3. STUDENT FOODS QROUP OUTLINE 



1. Introduce selves and explain reason for meeting. 

2. Introduce students 



Where from? 

How long in the U.S.? 

Whether they have family, and if so, who speaks English? 
Whether they are or have been employed in the U.S. ? 
Whether they have taken ESL classes in camp or in the U.S.? 

3. Now we would like to talk about what helps you learn English. 
Think about this class and about other classes you have been in. 

Probes: (To be used to stimulate conversation and to elicit 

response when students do not react well to the open- 
ended question on their own.) 



read 

listen and repeat 

talk to each other 

conversations 

use workbooks 

use dictionary 

have homework 

practice speaking outside 



write sentences 
copy sentences 
memorize 

role play 
spelling 
play games 
have tutors 



4. What would be the best way to set up classes for refugees? 
Is it important to; 

Probes: (To be used to stimulate conversation and to elicit 

response when students do not react well t othe open- 
ended question on their own.) 



Meet every day 
Have day care 

Have someone in school who can translate 

Have someone to help with job search 

Classes close to home 

Day or night classes 

Speakers on American life 

Assistance with resettlement problems 

5. What are the best things about your class? 

6. If you could change some, things about your class, what would you 
change? 
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APPENDIX C. 
S VALUATORS 9 BIOGRAPHIES 



DAVID NiuSOH 

4334 Grand Avenue South 
Minneapolis, Minnesota 55409 
(612) 822-7579 

David Nelson has been providing consulting and technical assistance 
on human service issues in the Twin Cities for the last ten years. 
Some examples of his past responsibilities include Director of 
Special Projects and Acting Executive Director of Enablers, a 
nonprofit agency that provided technical assistance, consulting 
service and public policy research for youth-related human services. 
In that capacity Mr* Nelson worked with dozens of agencies, providing 
them management, training or consulting services. 

Since 19Q r > Mr* Nelson has worked as an independent consultant, and 
has specialized on program design, evaluation and public policy 
research projects within the human service sector* 

The most recent responsibilities that relate directly to this project 
include the development of Refugees in Ramsey County * a 1984 report 
for The St. Paul Foundation* Mr* Nelson was one of the four-person 
team members responsible for the research and analysis* In addition, 
Mr* Nelson also prepared the 1985 research to The St* Paul 
Foundation, Southeast Asian Mutual Assistance Associations in 
Minnesota . These efforts have provided Mr* Nelson with a solid 
understanding of the refugee community and the issues that they face* 

Mr* Nelson's project evaluation experience has included both 
formative and summative projects, ranging from an evaluative review 
of all legal services for a local foundation, to evaluation of 
specific projects to assist in funding decisions* Most recent is a 9- 
month project with the Council on Quality Education to evaluate 
their Innovative/Cost Effective projects* 



HARQ BRINGEWATT 

268 South Milton 

St* Paul, Minnesota 55105 

612 222-4140 

Marg Bringewatt has worked for ten years in human service agencies. 
Direct service experience includes refugee resettlement, family and 
crisis intervention, emergency shelter, youth counseling, child care, 
neighborhood organizing, and community education* In addition Ms* 
Bringewatt worked for three years at the St* Paul Foundation where 
she reviewed over 600 proposals from local nonprofits, provided 
technical assistance to numerous agencies, and managed a variety of 
research projects* 



Her experience with refugees began in 1979 when she worked closely 
with refuge leaders and others to establish a crime prevention 
program in St. Paul Public Housing. At the St. Paul Foundation, Ms. 
Bringewatt has worked closely with several MAA's, wrote and published 
the monthly Refugee Newsletter which was a circulation of 1000, and 
has been a frequent public speaker on refugee issues. Two major 
projects included the production of the studies Refugees in Ramsey 
flwnty, and Southeast Asian Mutual Ass istance Associations in 
Minnesota, completed in 1984 and 1985. She is also a member of the 
Governor's State Refugee Advisory Council and works with the 
education subcommittee. 

In 1982 Ms. Bringewatt earned a Masters Degree in Public Affairs from 
the Humphrey Institute of Public Affairs, where she specialized in 
administration and community and economic development. Other public 
affairs activities have included organizing roundtable conferences 
with local "foundations, volunteer work for the Minnesota Project and 
membership in the Citizens' League. 

Ms. Bringewatt has worked with evaluation in a variety of settings. 
She designed a post-treatment evaluation system for a local nonprofit 
with the assistance of the Office of Research and Statistics at the 
Wilder Foundation. She has also assisted numerous grantees while at 
The St. Paul Foundation in creating evaluation systems that go beyond 
data collection to thoughtful analysis of outcomes. In addition she 
created a three-year program which will grant funds to nonprofits to 
conduct evaluation or research in the areas of intraf amilial sex 
abuse. Ms. Bringewatt also designed the internal evaluation to be 
used for this project. 
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